41 research outputs found
Modeling of Coupled Memristive-Based Architectures Applicable to Neural Network Models
This chapter explores the dynamic behavior of dual flux coupled memristor circuits in order to explore the uncharted territory of the fundamental theory of memristor circuits. Neuromorphic computing anticipates highly dense systems of memristive networks, and with nanoscale devices within such close proximity to one another, it is anticipated that flux and charge coupling between adjacent memristors will have a bearing upon their operation. Using the constitutive relations of memristors, various cases of flux coupling are mathematically modeled. This involves analyzing two memristors connected in composite, both serially and in parallel in various polarity configurations. The new behavior of two coupled memristors is characterized based on memristive state equations, and memductance variation represented in terms of voltage, current, charge and flux. The rigorous mathematical analysis based on the fundamental circuit equations of ideal memristors affirms the memristor closure theorem, where coupled memristor circuits behave as different types of memristors with higher complexity
Memristive Reservoirs Learn to Learn
Memristive reservoirs draw inspiration from a novel class of neuromorphic
hardware known as nanowire networks. These systems display emergent brain-like
dynamics, with optimal performance demonstrated at dynamical phase transitions.
In these networks, a limited number of electrodes are available to modulate
system dynamics, in contrast to the global controllability offered by
neuromorphic hardware through random access memories. We demonstrate that the
learn-to-learn framework can effectively address this challenge in the context
of optimization. Using the framework, we successfully identify the optimal
hyperparameters for the reservoir. This finding aligns with previous research,
which suggests that the optimal performance of a memristive reservoir occurs at
the `edge of formation' of a conductive pathway. Furthermore, our results show
that these systems can mimic membrane potential behavior observed in spiking
neurons, and may serve as an interface between spike-based and continuous
processes.Comment: 7 pages, 6 figures, ICONS 2023, accepte
SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks
As the size of large language models continue to scale, so does the
computational resources required to run it. Spiking Neural Networks (SNNs) have
emerged as an energy-efficient approach to deep learning that leverage sparse
and event-driven activations to reduce the computational overhead associated
with model inference. While they have become competitive with non-spiking
models on many computer vision tasks, SNNs have also proven to be more
challenging to train. As a result, their performance lags behind modern deep
learning, and we are yet to see the effectiveness of SNNs in language
generation. In this paper, inspired by the Receptance Weighted Key Value (RWKV)
language model, we successfully implement `SpikeGPT', a generative language
model with binary, event-driven spiking activation units. We train the proposed
model on two model variants: 45M and 216M parameters. To the best of our
knowledge, SpikeGPT is the largest backpropagation-trained SNN model to date,
rendering it suitable for both the generation and comprehension of natural
language. We achieve this by modifying the transformer block to replace
multi-head self attention to reduce quadratic computational complexity O(N^2)
to linear complexity O(N) with increasing sequence length. Input tokens are
instead streamed in sequentially to our attention mechanism (as with typical
SNNs). Our preliminary experiments show that SpikeGPT remains competitive with
non-spiking models on tested benchmarks, while maintaining 20x fewer operations
when processed on neuromorphic hardware that can leverage sparse, event-driven
activations
Memristive Stochastic Computing for Deep Learning Parameter Optimization
Stochastic Computing (SC) is a computing paradigm that allows for the
low-cost and low-power computation of various arithmetic operations using
stochastic bit streams and digital logic. In contrast to conventional
representation schemes used within the binary domain, the sequence of bit
streams in the stochastic domain is inconsequential, and computation is usually
non-deterministic. In this brief, we exploit the stochasticity during switching
of probabilistic Conductive Bridging RAM (CBRAM) devices to efficiently
generate stochastic bit streams in order to perform Deep Learning (DL)
parameter optimization, reducing the size of Multiply and Accumulate (MAC)
units by 5 orders of magnitude. We demonstrate that in using a 40-nm
Complementary Metal Oxide Semiconductor (CMOS) process our scalable
architecture occupies 1.55mm and consumes approximately 167W when
optimizing parameters of a Convolutional Neural Network (CNN) while it is being
trained for a character recognition task, observing no notable reduction in
accuracy post-training.Comment: Accepted by IEEE Transactions on Circuits and Systems Part II:
Express Brief
Side-channel attack analysis on in-memory computing architectures
In-memory computing (IMC) systems have great potential for accelerating
data-intensive tasks such as deep neural networks (DNNs). As DNN models are
generally highly proprietary, the neural network architectures become valuable
targets for attacks. In IMC systems, since the whole model is mapped on chip
and weight memory read can be restricted, the system acts as a "black box" for
customers. However, the localized and stationary weight and data patterns may
subject IMC systems to other attacks. In this paper, we propose a side-channel
attack methodology on IMC architectures. We show that it is possible to extract
model architectural information from power trace measurements without any prior
knowledge of the neural network. We first developed a simulation framework that
can emulate the dynamic power traces of the IMC macros. We then performed
side-channel attacks to extract information such as the stored layer type,
layer sequence, output channel/feature size and convolution kernel size from
power traces of the IMC macros. Based on the extracted information, full
networks can potentially be reconstructed without any knowledge of the neural
network. Finally, we discuss potential countermeasures for building IMC systems
that offer resistance to these model extraction attack
PowerGAN: A Machine Learning Approach for Power Side-Channel Attack on Compute-in-Memory Accelerators
Analog compute-in-memory (CIM) accelerators are becoming increasingly popular
for deep neural network (DNN) inference due to their energy efficiency and
in-situ vector-matrix multiplication (VMM) capabilities. However, as the use of
DNNs expands, protecting user input privacy has become increasingly important.
In this paper, we identify a security vulnerability wherein an adversary can
reconstruct the user's private input data from a power side-channel attack,
under proper data acquisition and pre-processing, even without knowledge of the
DNN model. We further demonstrate a machine learning-based attack approach
using a generative adversarial network (GAN) to enhance the reconstruction. Our
results show that the attack methodology is effective in reconstructing user
inputs from analog CIM accelerator power leakage, even when at large noise
levels and countermeasures are applied. Specifically, we demonstrate the
efficacy of our approach on the U-Net for brain tumor detection in magnetic
resonance imaging (MRI) medical images, with a noise-level of 20% standard
deviation of the maximum power signal value. Our study highlights a significant
security vulnerability in analog CIM accelerators and proposes an effective
attack methodology using a GAN to breach user privacy